Restricted Boltzmann machines (RBM) and its variants have become hot researchtopics recently, and widely applied to many classification problems, such ascharacter recognition and document categorization. Often, classification RBMignores the interclass relationship or prior knowledge of sharing informationamong classes. In this paper, we are interested in RBM with the hierarchicalprior over classes. We assume parameters for nearby nodes are correlated in thehierarchical tree, and further the parameters at each node of the tree beorthogonal to those at its ancestors. We propose a hierarchical correlated RBMfor classification problem, which generalizes the classification RBM withsharing information among different classes. In order to reduce the redundancybetween node parameters in the hierarchy, we also introduce orthogonalrestrictions to our objective function. We test our method on challengedatasets, and show promising results compared to competitive baselines.
展开▼
